It is not the public key, but the hypercore discovery key. This is an important detail. We never expose the public key over the network, as it is used as a capability

It's truncated because dns subdomains can at most be 63 chars also

I think it's big endian port

personal note that we should use blake2b here instead

secre -> secret

{DISCOVERY_KEY}.dat.local

{DISCOVERY_KEY}.dat.local

Good question, I think its 1-2min

Yes they should. They are GC'ed every 5-10 mins

discovery key

main thing leaked is who is talking to who (which is of course important). we never leak the capability (public key) so passive listeners cannot access data / decrypt data - they can also see Alice and Bob are talking to each other probably

it's important to indicate who that information is leaked to. elsewhere in the documentation (e.g. in the security FAQ) we are lead to believe that information is leaked only to the members of the swarm, which is not really accurate. sure, the contents are visible only to the members of the swarm, but metadata like public (and private?) IP addresses and relationships between people are spread out much more widely that I first believed when reviewing the protocol.

in particular, if i understand this DEP correctly, it implies that discoverN.datproject.org know precisely:

1. when a peer comes online (when Alice runs dat share)
2. when a peer looks for content (when Bob runs dat clone $ALICEHASH)
3. that Alice and Bob are related

This raises all sorts of privacy concerns which should be answered by the dat project. For example:

• does the discovery server keep logs?
• what is the retention policy?
• who has access to those logs?

I think the current section about Privacy concerns is great, but should be expanded to cover for this peculiar property of the protocol. The security FAQ should also be updated to mention this, but that's a separate issue: I've documented my concerns with that in dat-ecosystem-archive/docs#127

yea good idea. 32-chars.32-chars or use an encoding other than hex that is still dns friendly. Thoughts?

You could hash the discovery key to something like 63 bytes?

Smart

If we hash to 63 bytes (not change encoding) we're basically just losing a byte of specificity. Why not just do 32.32 and stick with hex? We could also switch to base32.

32.32 seems reasonable.

I'm not sure how I feel about encoding these function signatures here. I did something similar in the hyperdb DEP, but here it feels really specific to the existing dat implementation. Maybe just describe what semantics are necessary for lookup, and what additional semantics are necessary to announce/cancel membership or subscribe to a feed of peer updates?

I'm pretty sure we use discovery keys (not public keys) for all discovery mechanisms.

The discovery key is a BLAKE2b "keyed hash" of the string "hypercore" using the public key (32 bytes), described in the wire protocol DEP (WIP).

SRV requests are also possible. Eg, on the command line:

dig @discovery1.publicbits.org 905fd1b6504698425e8bec3dbb77d757e281d505.dat.local SRV

returns something like:

0 0 44113 172.19.0.4.

Which, IIRC, is port 44113 on host 172.19.0.4 (note the trailing period, which is not a typo).

I think 10 minutes, and clients should re-announce, but I don't have a reference.

Here are two other privacy concerns off the top of my head:

• the concern of being able to discovery (and potentially download) all content "discovered" via these mechanisms. This is mitigated by using discovery keys (instead of public keys) for download
• the ability to discover who has what content on the network (if you know a priori what content is associated with which discovery keys). Eg, imagine somebody sharing leaked documents; if the documents (dat archive, and thus discovery key) become public, somebody can make a list of all peers who have exchanged (or "knew of") that archive. I don't know any mitigation for this right now. This is also a concern with the wire protocol; in that case it could be mitigated by encrypting the entire transaction (including the discovery key verification), but not with the current encryption scheme.

You will receive a response that includes a full peer listing

If I go through a valid "probe" step, acquire a session token, and then announce multiple ports, that would seem to increase the full peer listing arbitrarily.

Since simple lookups do not require a token, then it should be possible for me to spoof the IP address in simple lookups and use (previously constructed) arbitrarily large full peer listing to execute DoS on a target.

Good find, this is a weakness cc @mafintosh

it's important to indicate who that information is leaked to. elsewhere in the documentation (e.g. in the security FAQ) we are lead to believe that information is leaked only to the members of the swarm, which is not really accurate. sure, the contents are visible only to the members of the swarm, but metadata like public (and private?) IP addresses and relationships between people are spread out much more widely that I first believed when reviewing the protocol.

in particular, if i understand this DEP correctly, it implies that discoverN.datproject.org know precisely:

1. when a peer comes online (when Alice runs dat share)
2. when a peer looks for content (when Bob runs dat clone $ALICEHASH)
3. that Alice and Bob are related

This raises all sorts of privacy concerns which should be answered by the dat project. For example:

• does the discovery server keep logs?
• what is the retention policy?
• who has access to those logs?

I think the current section about Privacy concerns is great, but should be expanded to cover for this peculiar property of the protocol. The security FAQ should also be updated to mention this, but that's a separate issue: I've documented my concerns with that in dat-ecosystem-archive/docs#127